In content-based publish/subscribe systems, publishers typically post information on the system and subscribers generally specify their interest in receiving certain information in terms of predicates on the posted information. Intermediate brokers, which may reside between the publishers and subscribers, typically filter information in order to forward only the relevant information that is requested to downstream subscribers connected to the intermediate brokers. In the course of forwarding information from a publisher to a subscriber, however, information can be lost or otherwise corrupted, often as a result of a system failure. System failures can occur in one or more of the publishers, subscribers, and/or intermediate brokers, or in one or more links connecting such entities with one another. Therefore, it is advantageous to provide a publish/subscribe system which incorporates features for protecting against such loss of information.
There are various motivations which exist for providing gapless delivery of information in the presence of system failures, including, for example: (1) service agreements (e.g., it is unacceptable for certain stock traders not to be able to access a trade event that others can access); and (2) message interdependencies, wherein messages may be used by a subscribing application to accumulate a view (e.g., a snapshot of a sporting event), where missing or reordered messages can cause an incorrect state to be displayed.
Conventional solutions to the problem of safeguarding information against system failures typically employ message queuing which utilizes store-and-forward routing protocols, wherein each intermediate broker in the publish/subscribe system must reconstruct a gapless information stream prior to forwarding messages to a downstream broker. Such conventional protocols, however, generally cannot support a high-throughput since the information stream is delayed when a broker recovers from a failure, or when a single message is lost. Existing message queuing systems typically implement store-and-forward routing protocols by logging a message to persistent storage at every intermediate broker in the system. This technique, however, consumes a significant amount of memory and requires increased system overhead, which in turn undesirably impacts system cost and speed.
Consequently, there exists a need for improved techniques for use in a publish/subscribe system, for providing gapless delivery of information to subscribers in the system, despite failures which may occur in the system.